Point cloud assisted photogrammetric rendering method and apparatus

ABSTRACT

A point cloud assisted photogrammetric restitution method is described. Said method comprises:
         the simultaneous visualization on a screen ( 5 ) of the ensemble of a stereoscopic image ( 33 ) and a point cloud ( 34 ) acquired on a given area ( 2 ), said stereoscopic image deriving from at least a couple of photogrammetric images ( 11 ) acquired on said given area ( 2 ) and oriented according to the same coordinate system of the point cloud,   the real time connection of the collimation mark (S) of the stereoscopic image with the corresponding collimation mark (S′) of the point cloud.

The present invention relates to a point cloud assisted photogrammetricrestitution (or plotting) method and apparatus thereof.

The use of photogrammetry, e.g. for the production of topographic maps,is known in the prior art.

Stereoscopic observation of pairs of images is normally used inphotogrammetry. Said images are digital images, i.e. acquired digitallyor acquired in analogue manner and then digitalized, the innerorientation features (i.e. the focus point, the main point and thedistortion parameters) and the outer orientation features (e.g. theimage shot centre and the absolute orientation angles for perspectiveshots) of which are known. By stereoscopically observing the images, theplotter operator performs a three-dimensional digital vectoring of thedetails to be plotted (e.g. road curb lines, building identificationlines, reference points, etc.). In practice, identified lines areextracted by means of point sequences and single points, the coordinatesof which are three-dimensional. For example, if the object is the roofof a house, the plotter operator extracts a continuous line according togiven rules, which line describes the dimensions of the roof, in whichthe vertex coordinates (the Cartesian coordinates X, Y and Z) aredetermined according to the orientation of the images.

Photogrammetry is also used in land and industrial, not only aerial,contexts.

A critical aspect of photogrammetry is the detail with which the depth(typically the Z coordinate) of an object is determined; it depends onthe B/H (base/height) ratio, where B is the distance between imagecenters O′ and O″, commonly known as image base, and H is the distancebetween the image base and the object P, as shown in FIG. 1. Inparticular, the image base is the distance which separates the two imagecenters, which are the points in space in which the two photographs of astereoscopic image are taken. The height H is the average height betweenthe image base B and the framed surface; in the aerial case, H is therelative flying height, i.e. the difference between the flying heightand the average height of terrain in that zone

In light of the prior art, it is the object of the present invention toprovide a method in which photogrammetric restitution is point cloudassisted to allow a more complete visualization of the area.

In accordance with the present invention, said object is reached bymeans of a point cloud assisted photogrammetric restitution method, saidmethod comprising:

the simultaneous visualization on a screen of the ensemble of astereoscopic image and a point cloud acquired on a given area, saidstereoscopic image deriving from at least a couple of photogrammetricimages acquired on said given area and oriented according to the samecoordinate system of the point cloud,

the real time connection of the collimation mark of the stereoscopicimage with the corresponding collimation mark of the point cloud.

In a preferred embodiment, the method according to the present inventionalso allows a more accurate, rapid determination of the depth (Zcoordinates) of the collimated points.

Again in accordance with the present invention, a point cloud assistedphotogrammetric restitution method is provided, as defined in claim 9.

By virtue of the present invention, a method is provided for easilydetermining misalignments between objects of the stereoscopic image andthe point cloud.

The features and the advantages of the present invention will beapparent from the following detailed description of a practicalembodiment thereof, shown by way of non-limitative example in theaccompanying drawings, in which:

FIG. 1 diagrammatically shows the arrangement of sensing means in anarea for acquiring images useful for the formation of the stereoscopicimage;

FIG. 2 is a block chart of the apparatus for the production of pointcloud assisted stereoscopic images according to the present invention,

FIGS. 3-5 show images which are obtained with the method according tothe invention.

FIG. 2 shows a block chart of the point cloud assisted photogrammetricrestitution apparatus according to the present invention.

The apparatus comprises computerized means 3 comprising a memory 31, inwhich a software program 100 is installed, a microprocessor 32 and ascreen 5; the software program 100 is capable of implementing a pointcloud assisted photogrammetric restitution method.

The method comprises the simultaneous visualization on a screen 5 of theensemble 50 of a stereoscopic image 33 and a point cloud 34 acquired ona given area 2. The stereoscopic image derives from at least a pair ofphotogrammetric images 11 acquired on said given area 2 and orientedaccording to the same coordinate system as the point cloud. The methodcomprises the real time connection of the collimation mark of thestereoscopic image 33 with the corresponding collimation mark of thepoint cloud 34.

The stereoscopic image derives from at least one pair of photogrammetricimages 11 acquired on the area 2. The images 11 are obtained by means ofsensing means 1 adapted to shoot the zone or areas 2 and are digitalimages; the images 11 may also be analogue images which are digitalized.

The device 3 comprises a memory 31, in which a software application 100is installed and runs; said memory communicates with a microprocessor32. The means 3 may also process images 11 to obtain a stereoscopicimage 33.

The point cloud 34 is preferably obtained by means of a laser scanner 4with LiDAR remote sensing technique, adapted to point the same zone 2;the point cloud 34 is an ensemble of points defined in three-dimensionalcoordinates of the zone 2. The acquisition of the images 11 and of thepoint cloud 34 may occur at the same time or in sequence after oneanother.

The point cloud 34 is oriented in a given manner on a coordinate system,e.g. on a Cartesian coordinate system X, Y and Z, and the images 11 mustbe oriented in the same coordinate system. Said stereoscopic image 33 istherefore aligned with the point cloud 34.

The software application 100 allows to visualize the ensemble 50 of thestereoscopic image 33 and of the point cloud 34 on a display 5; FIG. 3shows the ensemble of a stereoscopic image 33 of a landscape with ahouse 54 combined with the point cloud 34.

The software application 100 allows the real time connection of thecollimation mark S of the stereoscopic image 33 with the correspondingcollimation mark S′ of the point cloud 34. The collimation mark is thepoint either indicated or pointed by a pointer which can run on theimage 33 and on the point cloud 34, e.g. a cursor. The collision mark Sof the stereoscopic image 33 is coupled with the collimation mark S′ ofthe point cloud 34 so that all movements on the axes X, Y and Z of thecollimation mark S on the stereoscopic image 33 cause a equal movementof the collimation mark S′ of the point cloud 34. The lines and pointscreated by the user, which describe the geometry of the objects to beacquired (e.g. the roofs of the buildings in FIG. 3), can be visualizedon the stereoscopic image 33 and on the point cloud 34 in order toevaluate acquisition accuracy with regards to the stereoscopic image andthe point cloud. By means of the combined view of the stereoscopic image33 and the point cloud 34, misalignments between the images can beverified and the quality of the acquisition for both the stereoscopicimage and the point cloud can be measured.

The method in accordance with the invention comprises calculating inreal time the depth or Z coordinate of the collimation mark S of thestereoscopic image 33 according to the obtained depth of the point cloud34.

In this manner, the three-dimensional coordinates of the collimationmark S are immediately acquired because the software itself obtains thecoordinates X and Y, i.e. the planar coordinates of the collimation markS, from the stereoscopic image 33, while the depth or in this case thecoordinate Z, is automatically acquired from the point cloud 34. Forexample, if the collimation mark S is on the vertex of a roof of abuilding 54, the planar coordinates will be given by the stereoscopicimage 33 but the depth Z will be obtained automatically by the pointcloud 34.

Depth Z may be calculated by means of various algorithms.

A first algorithm is used to calculate depth Z by searching the pointcloud which is closest to the position of the collimation mark S′ withina given distance chosen by the user.

With another algorithm, alternative to the first, the software 100 mayplot a depth Z calculated on an averaging plane interpolating the pointsof the point cloud 34 which are closer to the position of thecollimation mark S′ within a given distance, again chosen by the user ona case-by-case basis.

With a third algorithm, alternative to the preceding ones, the softwareprogram 100 may plot a depth Z calculated according to the weightedaverage of the Z coordinates of the points of the point cloud which arewithin a given distance from the collimation mark S′, with weight equalto the inverse of the distance, which is again chosen by the user.

If the cloud points are classified according to the type of point (e.g.land, vegetation, artificial works, etc.), the user can select whichclasses to be included in the search and which are to be excluded.

For each algorithm, the point search within a given distance from thecollimation mark S′ may be two-dimensional (when only the planarcoordinates X, Y of the collimation mark S′ and of the cloud points areconsidered to calculate the distance) or three-dimensional (when theplanar coordinates and the depth, i.e. the current X, Y, Z coordinatesof the collimation mark S′ and the coordinates X, Y, Z of the cloudpoints are considered for calculating the distance).

In all cases, if the depth Z calculated in this manner is deemed notcorrect, the operator can always deactivate the function and manuallydetermine the position, based on the stereoscopic vision of the image33.

Thus, the normal conditions, the operator simply needs to move thecollimation mark S on the image 33 and the software program 100 plotsthe depth Z of the position of the collimation mark S in real time,according to the point cloud, on the basis of the set capture parameters(the search radius and the possible excluded point classes) and the setsearch algorithm.

The method according to the present invention allows easy interpretationof images details combined with accurate depth determined by the pointcloud.

The point cloud 34 may also be stereoscopically visualized by using apair of prospective projections, which have a given mutual parallax, asshown in FIG. 4.

The acquisition with combined view is must faster and more expedite thantraditional photogrammetric acquisition, in which the operator mustmanually determine the depth of the point and perceive that thecollimation mark rests on the surface.

Furthermore, it is much more accurate and reliable, because the depth Zis given by the point cloud and the accuracy of this data does notdepend on the ratio between height H and image base B and on theoperator's ability in defining the position of the depth Z of thecursor.

Finally, there is a much greater detail perception with respect to theuse of the point cloud only.

Furthermore, by having a stereoscopic view, the operator perceives inall cases the three-dimensional shape of the object being traced,therefore a possible accidental error, e.g. in capturing the point clouddepth Z, may be easily diagnosed and eliminated from the start.

The dimension of the image 33 and of the point cloud 34 on the screenmay be varied at will. With regards to the point cloud 34, the user canmodify the parallax and the field of view (FOV) to either emphasize thestereoscopic effect or reduce it to zero and can select all pointclasses to be visualized (in the case of points which also have theclassification attribute). Both the stereoscopic image 33 and the pointcloud 34 can display the lines and points created by the user, whichdescribe the geometry of the acquired objects (e.g. in FIG. 4, thestreet curbs), in order to evaluate the acquisition accuracy in bothcases (i.e. in relation to the stereoscopic image formed by the imagesand in relation to the point cloud). The movements of thethree-dimensional cursor which modify the scene in image 33 cause acorresponding change in the scene of the point cloud 34 so that they arecoupled from the view point of the framed detail.

In order to better understand details (e.g. the shape of a roof), theuser may represent the cloud points 34 with staggered scales(emphasizing the depth Z) and may turn the scene of the point cloud 34at will.

Typically, however, the depth accuracy of the point cloud is much higherthan what can be obtained with image pair due to the B/H ratio values(described above) which is used in photogrammetric images.

Therefore, in normal cases, in some cases very obvious differences oflevel of the point cloud 34 cannot be evaluated by the stereoscopicmodel formed by the images. This problem is overcome by this technique.

The fact of not needing to manually set the cursor position bystereoscopically observing the model formed by the images makes thistechnique much more expedite than the classic photogrammetric technique.

The normal sensing conditions (photogrammetric and lidar) make thistechnique also much more accurate in evaluating the depth than theclassic photogrammetric technique.

On the other hand, the point cloud alone is not sufficient to correctlydescribe given details (such as for example the edge of a building) asmay be easily done instead with the stereoscopic image.

The combined view allows in all case human control by the operator, whomay validate the result and/or modify it.

FIG. 5 shows an overview of this technique used to acquire the shape ofthe roof of the building. The two zoom levels of the stereoscopic image33 with different rotation angle of the point cloud 34 help toappreciate the geometric quality of the acquired eaves line.

1. A point cloud assisted photogrammetric restitution method, saidmethod comprising: the simultaneous visualization on a screen of theensemble of a stereoscopic image and a point cloud acquired on a givenarea, said stereoscopic image deriving from at least a couple ofphotogrammetric images acquired on said given area and orientedaccording to the same coordinate system of the point cloud, the realtime connection of the collimation mark of the stereoscopic image withthe corresponding collimation mark of the point cloud.
 2. The methodaccording to claim 1, characterized in that it comprises the calculationin real time of the depth of the collimation mark of the stereoscopicimage based on the depth obtained by the point cloud.
 3. The methodaccording to claim 2, characterized in that said calculation providesthe depth of the point of the point cloud that is nearer to thecollimation mark within a determined distance.
 4. The method accordingto claim 2, characterized in that said depth is calculated on aaveraging plane interpolating the points of the point cloud which arecloser to the collimation mark within a determined distance.
 5. Themethod according to claim 2, characterized in that said depth iscalculated on the base of the weighted average of the depths of thepoints of the point cloud which find themselves within a certaindistance of the collimation mark with weight equal to the inverse of thedistance.
 6. The method according to claim 3, characterized in that saidresearch of said at least one point of said point cloud that findsitself within a determined distance from the collimation mark occurs ina bidimensional way considering only the planar coordinates of said atleast one point.
 7. The method according to claim 3, characterized inthat said research of said at least one point of said point cloud thatfinds itself within a determined distance of the collimation mark occursin a tridimensional way considering the planar coordinates and the depthof said at least one point.
 8. The method according to claim 1,characterized in that it comprises the formation of a stereoscopic imageof the point cloud by means of the setting of a couple of perspectiveprojections having a certain parallax between them.
 9. A point cloudassisted photogrammetric restitution apparatus, said apparatuscomprising informatic means comprising a memory wherein a software isinstalled and a microprocessor and a screen, said software being able toimplement a point cloud assisted photogrammetric restitution method asdefined in any one of the preceding claims.
 10. The apparatus accordingto claim 9, characterized in that said point cloud is produced by alaser scanner with lidar remote sensing technique.